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(1) Real Party in Interest 

Google Inc., the assignee of record of the entire interest in this application, is the real 
party in interest. 

(2) Related Appeals and Interferences 

None. 

(3) Status of Claims 

Claims 109-156 are pending, with claims 109, 125, and 141 being independent. Claims 
109-156 have been rejected, and the rejections of all of these claims are appealed. 

(4) Status of Amendments 
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No amendments have been filed subsequent to the final rejection dated June 14, 2010. A 
listing of the claims is provided in the Appendix of Claims section. 

(5) Summary of Claimed Subject Matter 

In the discussion below, reference numerals and references to particular portions of the 
application are inserted for illustrative purposes only and are not meant to limit to the scope of 
the clams. The claimed subject matter is directed to methods and systems for modifying 
advertisement performance information for an advertisement based on concept performance 
information for a concept to which the advertisement corresponds. 

In some aspects, as recited in independent claim 109, advertisements that are available 
for presentation with a document are identified (page 29, lines 29-30 and page 33, lines 9-10). 
At least one processor receives initial advertisement performance information for the 
advertisements (page 29, lines 29-30; page 33, lines 10-12; and page 34, lines 21-25). The 
advertisement performance information for each advertisement specifies a measure of 
performance that is computed based on previous presentations of the advertisement (page 29, 
lines 29-30; page 33, lines 10-12; and page 34, lines 21-25). At least one processor identifies 
one or more concepts to which the document is relevant (page 11, lines 23-30; page 13, lines 20- 
30; page 33, lines 9-10; and page 34, lines 17-18). The concepts are identified based on content 
of the document (page 11, lines 23-30 and page 13, lines 20-30). At least one processor receives 
concept performance information for each identified concept for the document (page 30, lines 2- 
5 and page 34, lines 21-25). The concept performance information represents an aggregate 
performance of advertisements that were presented with the document and corresponding to the 
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concept (page 25, lines 5-29; page 33, lines 21-22; page 35, lines 1-15). At least one processor 
modifies the initial advertisement performance information for an advertisement (page 35, line 
19 - page 20, line 1 1). The initial advertisement performance information is modified using the 
concept performance information for an identified concept that corresponds to the advertisement 
(page 35, line 19 - page 20, line 1 1). At least one of the advertisements is forwarded based on 
the modified advertisement performance information (page 37, lines 7-9). 

In another aspect, as recited in independent claim 125, an apparatus includes one or more 
processors that are connected to a storage device (page 37, line 23 - page 38, line 9). The 
processors are configured to identify advertisements that are available for presentation with a 
document (page 29, lines 29-30 and page 33, lines 9-10) and to receive initial advertisement 
performance information for the advertisements (page 29, lines 29-30; page 33, lines 10-12; and 
page 34, lines 21-25). The advertisement performance information for each advertisement 
specifies a measure of performance computed based on previous presentations of the 
advertisement (page 29, lines 29-30; page 33, lines 10-12; and page 34, lines 21-25). The 
processors are also configured to identify one or more concepts to which the document is 
relevant (page 11, lines 23-30; page 13, lines 20-30; page 33, lines 9-10; and page 34, lines 17- 
18), to identify the concepts based on content of the document (page 11, lines 23-30 and page 13, 
lines 20-30), and to receive concept performance information for each identified concept for the 
document (page 30, lines 2-5 and page 34, lines 21-25). The concept performance information 
represents an aggregate performance of advertisements that were presented with the document 
and that correspond to the concept (page 25, lines 5-29; page 33, lines 21-22; page 35, lines 1- 
15). The processors are further configured to modify the initial advertisement performance 



Applicant 
Serial No. 
Filed 
Page 



Yingwei Claire Cui et al. 
10/649,585 
August 27, 2003 
4 of 34 



Attorney's Docket No.: 161 13-0760001 



information for an advertisement (page 35, line 19 - page 20, line 1 1), to modify the initial 
advertisement performance information using the concept performance information for an 
identified concept corresponding to the advertisement (page 35, line 19 - page 20, line 1 1), and 
to forward at least one of the advertisements based on the modified advertisement performance 
information (page 37, lines 7-9). 

In another aspect, as recited in independent claim 141, a storage device stores a computer 
program that includes one or more code segments (page 37, line 23 - page 38, line 9). When 
executed, the code segments cause at least one processor to identify advertisements that are 
available for presentation with a document (page 29, lines 29-30 and page 33, lines 9-10) and to 
receive initial advertisement performance information for the advertisements (page 29, lines 29- 
30; page 33, lines 10-12; and page 34, lines 21-25). The advertisement performance information 
for each advertisement specifies a measure of performance computed based on previous 
presentations of the advertisement (page 29, lines 29-30; page 33, lines 10-12; and page 34, lines 
21-25). The code segments also cause the processor to identify one or more concepts to which 
the document is relevant (page 11, lines 23-30; page 13, lines 20-30; page 33, lines 9-10; and 
page 34, lines 17-18). The concepts are identified based on content of the document (page 11, 
lines 23-30 and page 13, lines 20-30). The code segments also cause the processor to receive 
concept performance information for each identified concept for the document (page 30, lines 2- 
5 and page 34, lines 21-25). The concept performance information for each concept represents 
an aggregate performance of advertisements that were presented with the document and that 
correspond to the concept (page 25, lines 5-29; page 33, lines 21-22; page 35, lines 1-15). The 
code segments further cause the processor to modify the initial advertisement performance 
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information for an advertisement (page 35, line 19 - page 20, line 1 1). The initial advertisement 
performance information is modified using the concept performance information for an 
identified concept that corresponds to the advertisement (page 35, line 19 - page 20, line 1 1). 
The code segments also cause the processor to forward at least one of the advertisements based 
on the modified advertisement performance information (page 37, lines 7-9). 

(6) Grounds of Rejection to be Reviewed on Appeal 

Claims 109-156 stand rejected under 35 U.S.C. § 103(a) based on Gerace (U.S. Patent No. 
5,848,396) in view of Graham (U.S. Patent Publication No. 20060122884). 

(7) Argument 

I. Gerace, Graham and asserted combinations thereof fail to describe or suggest 
modifying the initial advertisement performance information, as recited by claim 109, and as 
similarly recited by independent claims 125 and 141 

Appellant requests reversal of the rejection of claim 109 and its dependent claims 110- 
124 because Gerace, Graham, and asserted combinations thereof fail to describe or suggest 
"modifying, by at least one processor, the initial advertisement performance information for an 
advertisement using the concept performance information for an identified concept 
corresponding to the advertisement," as recited by claim 109. 

The claimed modification of advertisement performance information is advantageous, for 
example, "because ads (or ad groups) and/or advertisers may be transient, in which case it may 
be difficult, if not impossible, to gather statistically significant amount of user behavior data with 



Applicant 
Serial No. 
Filed 
Page 



Yingwei Claire Cui et al. 
10/649,585 
August 27, 2003 
6 of 34 



Attorney's Docket No.: 161 13-0760001 



respect to a given ad (or ad group) for a given document. Since there may be a relatively small 
number of tracked user behavior (e.g., clicks) compared to the number of documents (as 
identified by their URLs) and ads, a user behavior (click) statistics matrix may be rather sparse. 

Some ads have very few clicks and impressions " However, the limited data points can be 

used with "performance parameter estimation (extrapolation) operations 496, 596 to populate 
user behavior (e.g., click) statistics of ads for which there is no (or very little) user behavior data 
for the document (or host). These operations 496, 596 may use concepts as a bridge for 
propagating statistics from ads to [other] ads." Application at 34. 1 The relied-upon art does not 
disclose or suggest such functionality for propagating statistics to ads having little or no user 
behavior data for the document, nor does it disclose or suggest the subject matter of the claims 
consistent with this functionality. 

At pages 2 and 3 of the Final Office Action of June 14, 2010 ("final action"), the 
Examiner states that "Gerace discloses a method, comprising . . . modifying ... the advertisement 
performance information for at least one of the one or more advertisements using the determined 
content concept performance information for the content concept associated with the at least one 
of the one or more advertisements (18:10-26)." 

At column 18, lines 1-26 Gerace recites: 

As discussed above, sponsors have the ability to place ads according to 
demographic profile. To do so, advertisers/sponsors complete a template, 
(preferably in the Ad Series Objects 33c) which allows them to list certain criteria 
as required, and to weight other criteria by importance. To ensure ads are shown 
to the appropriate target users, the sponsor then selects a minimum total weight 

1 In referring to the specification above, this brief does not intend to limit the scope of the claims 
to the example implementations shown in the drawings and described in the specification. 
Rather, the entitlement to have the claims interpreted broadly, to the maximum extent permitted 
by statute, regulation and applicable case law, is expressly affirmed. 
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which a user's demographic/psychographic profile must achieve before the 
advertisement is shown to the user. 

To ensure that sponsors achieve the optimal result from the ads they place, 
program 3 1 combines regression analysis with the above weighting technique to 
achieve real-time, automatic optimization as discussed previously. Under this 
auto-targeting system, an ad package is shown to general users. After a large 
number (e.g., 10,000) hits, program 31 runs a regression on a subject Ad 
Package Object 33b to see what characteristics are important, and who (type of 
user profile) the ad appeals to the most. Program 3 1 then automatically enters 
weighting information based on that regression to create a targeted system and 
runs the advertisement (Ad Package Object 33b) again in front of this new 
targeted group. Program 3 1 then runs a regression every 1 0,000 hits, for example, 
including a group of 500 general people as a control, and adjusts the weighting. 
This continues until the Ad Package is exhausted (i.e., the number of hits and 
click throughs are achieved. 

This portion of Gerace does not describe or suggest "modifying, by at least one 
processor, the initial advertisement performance information for an advertisement using the 
concept performance information for an identified concept corresponding to the advertisement." 
At most, this portion of Gerace discloses that "after a large number (e.g., 10,000) hits, program 
31 runs a regression on a subject Ad Package Object 33b [i.e., an advertisement] to see what 
characteristics are important, and who (type of user profile) the ad appeals to the most. Program 
3 1 then automatically enters weighting information based on that regression to create a targeted 
system and runs the advertisement." 

As described by Gerace, advertisers are provided "the ability to place ads according to 
demographic profile" by "completing a template, (preferably in the Ad Series Objects 33c) 
which allows them to list certain criteria as required, and to weight other criteria by importance." 
At column 12, lines 28-31, Gerace notes that "the sponsor specifies in Ad Series Object 33c the 
required and/or preferred psychographic and/or demographic criteria and relative importance 
(e.g., weight) with respect to each criterion." Thus, the criteria that are weighted by Gerace are 
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"psychographic and demographic criteria" for targeting users. Therefore, the "automatically 
enter[ed] weighting information" of Gerace is weighting information for "psychographic and 
demographic criteria," which is not the same as the "initial advertisement performance 
information." This is because the "psychographic and demographic criteria" of Gerace do not 
specify "a measure of performance computed based on previous presentations of the 
advertisement," as recited by claim 109. Rather, the "psychographic and demographic criteria" 
represent "characteristics of users." Gerace at Col. 2, line 47. One of ordinary skill in the art 
would readily appreciate that characteristics of users do not measure the performance of 
advertisements. Thus, Gerace' s modification of the relative importance of the psychographic 
and demographic targeting criteria do not constitute modification of "the initial advertisement 
performance information for an advertisement using concept performance information for an 
identified concept corresponding to the advertisement," as recited by claim 109. 

At page 5 of the final action, the Examiner states that "it is obvious that Gerace can track 
performance for different page content/concepts/topics and ad content relevance," and that "one 
would be motivated to do this to better target relevant ads." Even if the Examiner's 
interpretation of Gerace were proper (which appellant does not concede), tracking advertisement 
performance is not the same as "modifying ... the initial advertisement performance information 
for an advertisement using the concept performance information for an identified concept 
corresponding to the advertisement," as recited by claim 109. 

According to claim 109, the "advertisement performance information for each 
advertisement specifies] a measure of performance computed based on previous presentations 
of the advertisement," and it is this advertisement performance information that is modified by 
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the concept performance information. According to claim 109, the concept performance 
information represents "an aggregate performance of advertisements that were presented with 
the document and corresponding to the concept." The concept performance information for a 
document represents, on a per-concept basis, the aggregate performance of multiple 
advertisements that were presented with the document. According to claim 109, the initial 
advertisement performance information for a particular advertisement (i.e., a measure of 
performance that is computed based on previous presentations of the particular advertisement) is 
modified using the concept performance information. Thus, the initial advertisement 
performance information for the particular advertisement can be modified even if the particular 
advertisement has not been presented with the document (e.g., based on changes in the aggregate 
measure of performance for other advertisements that were presented with the document). One 
of ordinary skill in the art would readily appreciate that merely tracking advertisement 
performance, as suggested by the Examiner in the final action, would not result in a similar 
modification. 

In the final action, the Examiner noted that Graham "discloses concept/topic for 
documents and presenting relevant ads (Fig. lc; 9a; claim 27) and also relevance scores ([32, 39, 
47-49])." Even if the Examiner's interpretations of the cited portions of Graham were proper 
(which appellant does not concede), Graham still fails to cure the deficiencies of Gerace, as 
described above, because the disclosure of presenting relevant ads and the disclosure of 
relevance scores are not the same as "modifying . . . initial advertisement performance 
information for an advertisement" using "aggregate performance of advertisements that were 
presented with the document" and that correspond to the concept. For example, none of the 
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relied-upon portions of Graham disclose performance measures (e.g., click-through-rates) for 
advertisements that have been presented with a particular document. Rather, the relied-upon 
portions of Graham note a manner in which concepts of an advertisement can be compared to 
concepts associated with documents and/or concepts associated with a user to determine concept 



Claim 27 of Graham recites: 

An apparatus for determining if an advertising [sic] is relevant to a web 
document, the apparatus comprising: 

at least one memory having program instructions; 

at least one processor configured to execute the program instructions to 
perform operations of: 

identifying one or more advertisement-related concepts corresponding to 
the advertisement; 

analyzing content of the web document to identify one or more document- 
related concepts for the web document; 

comparing the one or more advertisement-related concepts to the one or 
more document-related concepts to determine if a match exists; and 

determining that the advertising is relevant to the web document if a match 



According to claim 27, "advertisement-related concepts" are compared to "document- 
related concepts to determine if a match exists," and a determination is made "that the 
advertising is relevant to the web document if a match exists." Graham's "comparing . . . 
advertisement-related concepts to . . . document-related concepts to determine if a match exists," 
even when combined with the disclosure of Gerace, does not result in the elements of claim 109. 
For example, the combination of Gerace and Graham still fails to disclose "modifying, by at 
least one processor, the initial advertisement performance information for an advertisement 
using the concept performance information for an identified concept corresponding to the 
advertisement," as recited by claim 109 because claim 27 fails to provide a suggestion that 
advertisement performance information is modified. 



similarity. 



exists. 
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The Examiner has also cited paragraphs 32, 39, and 47-49 of Graham in support of the 

rejection of claim 109. Paragraph 0032 recites: 

[0032] FIG. 1C illustrates a simplified diagram of a representative search 
technique in a particular embodiment according to the present invention. This 
diagram is merely an example which should not limit the scope of the claims 
herein. One of ordinary skill in the art would recognize many other variations, 
alternatives, and modifications. FIG. 1C shows a first portion of an example 
document 105, viewable by a user. Document portion 105 includes several 
highlighted words. These highlighted words comprise a context that causes the 
display of a first advertisement 107, "Ad 345" because of the correspondence 
between the concepts in this advertisement and the concepts in the contents of the 
document portion 105. Later, the user views a new area of the document, 
document portion 109. These highlighted words comprise a context that causes 
the display of a second advertisement 111, "Ad 938" because it is now the most 
relevant advertisement to the concepts being viewed by the user. If the use of only 
the highlighted words does not yield an advertisement with a sufficient relevancy 
score, Rj, other phrases contained in the viewing area can be used as the matching 
criteria for the collection of advertising concepts. . . . 

Paragraph 0032 of Graham notes, for example, that specified words on a document 
"comprise a context that causes display of the first advertisement 107, 'Ad345' because of the 
correspondence between the concepts in this advertisement and the concepts in the contents 
of the document portion 105." Paragraph 0032 describes selection of advertisements based on 
their relevancy "to the concepts being viewed by the user," but paragraph 0032 fails to teach or 
suggest that "advertisement performance information . . . specifying a measure of performance 
[and] computed based on previous presentations of the advertisement" is modified "using 
concept performance information" that represents "an aggregate performance of advertisements 
that were presented with the document and corresponding to the concept." 

Paragraph 0039 similarly fails to cure the deficiencies of Gerace. Paragraph 0039 
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[0039] In a particular representative embodiment according to the present 
invention, advertising can be targeted based upon the user's concepts of interest 
and how relevant these concepts are for a particular document. Relevance can be 
determined using a scoring or other method as described below. This information, 
in conjunction with the actual content of the document, can be used to target 
advertising to users. Concepts, which define sets of interests, can be collected 
from user input or other mechanisms. In a presently preferable embodiment, user 
privacy can be maintained because user profiles are not shared with advertisers. 
User profiles can be stored locally on the client computer, or at an ISP server, or a 
proxy service, for example. In some specific embodiments, advertisers can be 
provided statistical information recorded by the user's browser, for example. . . . 

According to paragraph 0039, "advertising can be targeted based upon the user's 
concepts of interest and how relevant these concepts are for a particular document." However, 
targeting advertisements based on relevance to a user's concepts of interest does not teach, 
describe, or suggest modification of advertisement performance measures for an advertisement 
using "aggregate performance of advertisements that were presented with the document" and that 
correspond to the concept. 

Paragraphs 0047-0049 also fail to cure the deficiencies of Gerace. Paragraphs 0047-0049 



[0047] The ci value provides a bias to the Sy value towards higher scoring 
user concepts. Given all relevant user concepts and all relevant advertising 
concepts, a matrix can be constructed having similarity values for each 
combination of user/advertising concepts. Table 1 illustrates a representative 
example of such a matrix in one specific embodiment. 



recite: 



Applicant : Yingwei Claire Cui et al. Attorney's Docket No.: 16113-0760001 

Serial No. : 10/649,585 

Filed : August 27, 2003 

Page : 13 of 34 



TABLE 1 



Comparison matrix 



Concept/ Ad 
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Ci 


Relevancy 
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Si, 2 


S 2,2 
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A 3 


Si, 3 


S 2 ,3 


s 3 , 3 ... 


s m , 3 
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Aj 


Si, n 


S 2 ,n 


si.*. . ; . 


s^ n 


Rn 



[0048] In this representative example, a collection of the relevant user 
concepts, {Q, C2, C3, . . . , Ci}, and relevant advertising concepts, {Aj, A 2 , A3, . . 
. , Aj} are included. The individual similarity value is represented for each 
comparison between user concept and advertising concept as Sy, where i is 
the user concept index and j is the advertising concept index. Next, in a step 424, 
a "best" advertising concept given all of the user concepts can be determined. 
In this step, the comparison value generated in step 406 of FIG. 4A between 
advertising and document, aj, is used to bias the user/advertising concept 
comparison value Sy towards advertising concepts which are more similar to 
the current document. This combination for each advertising concept yields the 
final similarity measure used to rank the advertisements: 




[0049] If multiple advertisements with the same Rj value are present, the 
aj value can be used to "break the tie," and the advertising concept with the higher 
score with respect to the document will be delivered to the user's browser. If Rj 
does not exceed some threshold, e.g. 20% relevancy, the advertisements can be 
ranked using the original content based value aj. In this case, the better solution is 
to deliver an advertisement that is at least similar in content to the current 
document. Note that in the case where neither the Rj value nor the aj value surpass 
a threshold, then either no advertisement is displayed, or other information in the 
advertisement area such as relevant headline news or news related to the user's 
profile can be displayed. 

Paragraphs 0047-0049 note that advertisement concepts and user concepts can be 



compared to select an advertisement that is associated with "advertising concepts which are more 
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similar to the current document." However, this portion of Graham still fails to disclose that a 
performance measure (e.g., an estimated click through rate) for an advertisement is modified 
using the "aggregate performance of advertisements that were presented with the document" and 
that correspond to the concept. 

The foregoing demonstrates that the disclosure of paragraphs 0032, 0039, and 0047-0049 
of Graham fail to cure the deficiencies of Gerace. Additionally, neither Gerace nor Graham 
provides any reason to modify the combination of Gerace and Graham to arrive at the subject 
matter of claim 109. 

For at least the reasons provided above, Gerace, Graham, and combinations thereof fail to 
describe or suggest "modifying, by at least one processor, the initial advertisement performance 
information for an advertisement using the concept performance information for an identified 
concept corresponding to the advertisement," as recited by independent claim 109. Accordingly, 
reversal of the rejections of claims 109-124 is requested. 

Independent claim 125 recites features similar to those discussed above with respect to 
independent claim 109. Specifically, claim 125 recites "one or more processors that are 
configured to . . . modify the initial advertisement performance information for an advertisement 
using the concept performance information for an identified concept corresponding to the 
advertisement." Accordingly, for reasons similar to those set forth above with respect to claim 
109, reversal of the rejection of claim 125 and its dependent claims is requested. 

Independent claim 145 recites features similar to those discussed above with respect to 
independent claim 109. Specifically, claim 141 recites "a storage device storing a computer 
program comprising one or more code segments that, when executed, cause at least one 
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processor to . . . modify the initial advertisement performance information for an advertisement 
using the concept performance information for an identified concept corresponding to the 
advertisement." Accordingly, for reasons similar to those set forth above with respect to claim 
109, reversal of the rejection of claim 141 and its dependent claims is requested. 

II. Conclusion and Relief 

For at least the reasons provided above, appellant respectfully requests reversal of the 
pending rejections. 

The Appeal Brief filing fee is paid concurrently herewith on the Electronic Filing System 
(EFS) by way of deposit account authorization. Please apply any other charges or credits to 
Deposit Account No. 06-1050. 



Respectfully submitted, 




Tracy M. Hitt 
Reg. No. 62,371 
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Appendix of Claims 



1-108. (Cancelled) 

109. (Previously Presented) A method, comprising: 

identifying advertisements that are available for presentation with a document; 

receiving, by at least one processor, initial advertisement performance information for the 
advertisements, the advertisement performance information for each advertisement specifying a 
measure of performance computed based on previous presentations of the advertisement; 

identifying, by at least one processor, one or more concepts to which the document is 
relevant, concepts being identified based on content of the document; 

receiving, by at least one processor and for each identified concept for the document, 
concept performance information representing an aggregate performance of advertisements that 
were presented with the document and corresponding to the concept; 

modifying, by at least one processor, the initial advertisement performance information 
for an advertisement using the concept performance information for an identified concept 
corresponding to the advertisement; and 

forwarding, based on the modified advertisement performance information, at least one 
of the advertisements. 



110. (Previously Presented) The method of claim 109, wherein the advertisement 
performance information includes selection information and impression information for the 
advertisements. 
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111. (Previously Presented) The method of claim 109, wherein the document is a Web 
page identified by a URL. 

112. (Previously Presented) The method of claim 109, wherein modifying the initial 
advertisement performance information for an advertisement comprises: 

selecting an identified concept to which the advertisement corresponds; and 
receiving concept performance information for the selected concept, the content 
performance information being computed based on performance of other advertisements that 
were presented with the document and correspond to the selected concept. 

113. (Previously Presented) The method of claim 1 09, wherein receiving, for each 
identified concept for the document, the concept performance information comprises 
determining, for each identified concept for the document, concept performance information 
using document specific advertisement performance information for the identified 
advertisements corresponding to the identified concept. 

1 14. (Previously Presented) The method of claim 1 1 3, further comprising: 
determining, for each identified concept for the document, document specific 

advertisement performance information for a first advertisement and a second advertisement; and 

determining, for each identified concept for the document, concept performance 
information using the document specific advertisement performance information for the first 
advertisement and the second advertisement corresponding to the identified concept. 



Applicant 
Serial No. 
Filed 
Page 



Yingwei Claire Cui et al. 
10/649,585 
August 27, 2003 
18 of 34 



Attorney's Docket No.: 161 13-0760001 



115. (Previously Presented) The method of claim 1 09, wherein the concept 
performance information for a concept includes one or more of selection information and 
impression information for advertisements presented with the document and corresponding to the 
concept. 

116. (Previously Presented) The method of claim 109, further comprising determining 
a confidence measure for the advertisement performance information, the confidence measure 
representing a confidence level for the advertisement performance information relative to other 
advertisement performance information. 

1 1 7. (Previously Presented) The method of claim 1 1 6, wherein determining the 
confidence measure for the advertisement performance information comprises determining a 
confidence measure for the advertisement performance information based on one or more of the 
age of data included in the advertisement performance information and the amount of the data 
included in the advertisement performance information. 

118. (Previously Presented) The method of claim 109, wherein the advertisement for 
which the initial advertisement performance information is modified is included with the 
forwarded advertisements. 
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1 19. (Previously Presented) The method of claim 109, wherein the advertisement for 
which the initial advertisement performance information is modified is different from the 
forwarded advertisements. 

120. (Previously Presented) The method of claim 1 09, wherein forwarding, based on 
the modified advertisement performance information, at least one of the advertisements 
comprises: 

comparing the modified advertisement performance information for an 
advertisement to a threshold; 

determining that the modified advertisement performance information exceeds the 

threshold; and 

forwarding the advertisement based on the modified advertisement performance 
information exceeding the threshold. 

121. (Previously Presented) The method of claim 109, wherein: 

identifying the one or more content concepts to which the document is relevant comprises 
identifying a first concept and a second concept to which the document is relevant; and 

the method further comprising determining that an advertisement is associated with the 
first concept and the second concept for the document. 



122. (Previously Presented) The method of claim 109, wherein: 
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receiving the initial advertisement performance information for the advertisements 
comprises receiving advertisement targeting information for the advertisements; and 

receiving the concept performance information comprises receiving concept targeting 
performance information representing an aggregate performance of advertisements targeted 
according to the advertisement targeting information that were presented with the document 
based on the advertisement targeting information. 

123. (Previously Presented) The method of claim 109, further comprising: 
identifying a first advertisement that does not have initial advertisement performance 

information; 

determining that at least one of the identified concepts for the document correspond to the 
first advertisement; 

receiving concept performance information for the at least one of the one identified 
concepts corresponding to the first advertisement; and 

determining, based on the received concept information for the at least one of the 
identified concepts associated with the first advertisement, estimated advertisement performance 
information for the first advertisement. 

124. (Previously Presented) The method of claim 123, wherein determining the 
estimated advertisement performance information for the first advertisement comprises 
determining a weighted-sum of the content concept information for the at least one of the 
identified concepts corresponding to the first advertisement. 
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125. (Previously Presented) An apparatus comprising one or more processors 
connected to a storage device, wherein the one or more processors are configured to: 

identify advertisements that are available for presentation with a document; 

receive initial advertisement performance information for the advertisements, the 
advertisement performance information for each advertisement specifying a measure of 
performance computed based on previous presentations of the advertisement; 

identify one or more concepts to which the document is relevant, concepts being 
identified based on content of the document; 

receive, for each identified concept for the document, concept performance information 
representing an aggregate performance of advertisements that were presented with the document 
and corresponding to the concept; 

modify the initial advertisement performance information for an advertisement using the 
concept performance information for an identified concept corresponding to the advertisement; 
and 

forward, based on the modified advertisement performance information, at least one of 
the advertisements. 

126. (Previously Presented) The apparatus of claim 125, wherein the advertisement 
performance information includes selection information and impression information for 
advertisements. 
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127. (Previously Presented) The apparatus of claim 125, wherein the document is a 
Web page identified by a URL. 

128. (Previously Presented) The apparatus of claim 125, wherein the one or more 
processors are configured to modify the initial advertisement performance information for an 
advertisement by: 

selecting an identified concept to which the advertisement corresponds; and 
receiving concept performance information for the selected concept, the content 
performance information being computed based on performance of other advertisements that 
were presented with the document and correspond to the selected concept. 

129. (Previously Presented) The apparatus of claim 125, wherein the one or more 
processors are configured to receive, for each identified concept for the document, the concept 
performance information comprises determining, for each identified concept for the document, 
concept performance information using document specific advertisement performance 
information for the identified advertisements corresponding to the identified concept. 

130. (Previously Presented) The apparatus of claim 129, wherein the one or more 
processors are further configured to: 

determine, for each identified concept for the document, document specific advertisement 
performance information for a first advertisement and a second; and 
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determine, for each identified concept for the document, concept performance 
information using the document specific advertisement performance information for the first 
advertisement and the second advertisement corresponding to the identified concept. 

131. (Previously Presented) The apparatus of claim 125, wherein the concept 
performance information for a concept includes one or more of selection information and 
impression information for advertisements presented with the document and corresponding to the 
concept. 

132. (Previously Presented) The apparatus of claim 125, wherein the one or more 
processors are further configured to determine a confidence measure for the advertisement 
performance information, the confidence measure representing a confidence level for the 
advertisement performance information relative to other advertisement performance information. 

133. (Previously Presented) The apparatus of claim 1 32, wherein the one or more 
processors are configured to determine the confidence measure for the advertisement 
performance information by determining a confidence measure for the advertisement 
performance information based on one or more of the age of data included in the advertisement 
performance information and the amount of the data included in the advertisement performance 
information. 
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134. (Previously Presented) The apparatus of claim 125, wherein the advertisement for 
which the initial advertisement performance information is modified is included with the 
forwarded advertisements. 

135. (Previously Presented) The apparatus of claim 125, wherein the advertisement for 
which the initial advertisement performance information is modified is different from the 
forwarded advertisements. 

136. (Previously Presented) The apparatus of claim 125, wherein the one or more 
processors are configured to forward, based on the modified advertisement performance 
information, at least one of the advertisements by: 

comparing the modified advertisement performance information for an 
advertisement to a threshold; 

determining that the modified advertisement performance information exceeds the 

threshold; and 

forwarding the advertisement based on the modified advertisement performance 
information exceeding the threshold. 

137. (Previously Presented) The apparatus of claim 125, wherein the one or more 
processors are configured to: 

identify the one or more content concepts to which the document is relevant by 
identifying a first concept and a second concept to which the document is relevant; and 
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determine that an advertisement is associated with the first concept and the second 
concept for the document. 

138. (Previously Presented) The apparatus of claim 125, wherein the one or more 
processors are configured to: 

receive the initial advertisement performance information for the advertisements by 
receiving advertisement targeting information for the advertisements; and 

receive the concept performance information by receiving concept targeting performance 
information representing an aggregate performance of advertisements targeted according to the 
advertisement targeting information that were presented with the document based on the 
advertisement targeting information. 

139. (Previously Presented) The apparatus of claim 125, wherein the one or more 
processors are further configured to: 

identify a first advertisement that does not have initial advertisement performance 
information; 

determine that at least one of the identified concepts for the document correspond to the 
first advertisement; 

receive concept performance information for the at least one of the one identified 
concepts corresponding to the first advertisement; and 
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determine, based on the received concept information for the at least one of the identified 
concepts associated with the first advertisement, estimated advertisement performance 
information for the first advertisement. 

140. (Previously Presented) The apparatus of claim 139, wherein the one or more 
processors are configured to determine the estimated advertisement performance information for 
the first advertisement by determining a weighted-sum of the content concept information for the 
at least one of the identified concepts corresponding to the first advertisement. 

141 . (Previously Presented) A storage device storing a computer program comprising 
one or more code segments that, when executed, cause at least one processor to: 

identify advertisements that are available for presentation with a document; 

receive initial advertisement performance information for the advertisements, the 
advertisement performance information for each advertisement specifying a measure of 
performance computed based on previous presentations of the advertisement; 

identify one or more concepts to which the document is relevant, concepts being 
identified based on content of the document; 

receive, for each identified concept for the document, concept performance information 
representing an aggregate performance of advertisements that were presented with the document 
and corresponding to the concept; 
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modify the initial advertisement performance information for an advertisement using the 
concept performance information for an identified concept corresponding to the advertisement; 
and 

forward, based on the modified advertisement performance information, at least one of 
the advertisements. 

142. (Previously Presented) The storage device of claim 141, wherein the 
advertisement performance information includes selection information and impression 
information for advertisements. 

143. (Previously Presented) The storage device of claim 141 , wherein the document is 
a Web page identified by a URL. 

144. (Previously Presented) The storage device of claim 141 , wherein the one or more 
code segments that, when executed, cause the at least one processor to modify the initial 
advertisement performance information for an advertisement comprises one or more code 
segments that, when executed, cause the at least one processor to: 

select identify an identified concept to which the advertisement corresponds; and 
receive concept performance information for the selected concept, the content 
performance information being computed based on performance of other advertisements that 
were presented with the document and correspond to the selected concept. 



Applicant : Yingwei Claire Cui et al. Attorney's Docket No.: 16113-0760001 

Serial No. : 10/649,585 
Filed : August 27, 2003 
Page : 28 of 34 

145. (Previously Presented) The storage device of claim 141 , wherein the one or more 
code segments that, when executed, cause the at least one processor to receive, for each 
identified concept for the document, the concept performance information comprise one or more 
code segments that, when executed, cause the at least one processor to determine, for each 
identified concept for the document, concept performance information using document specific 
advertisement performance information for the identified advertisements corresponding to the 
identified concept. 

146. (Previously Presented) The storage device of claim 145, wherein the one or more 
code segments that, when executed, cause the at least one processor to: 

determine, for each identified concept for the document, document specific advertisement 
performance information for a first advertisement and a second advertisement; and 

determine, for each identified concept for the document, concept performance 
information using the document specific advertisement performance information for the first 
advertisement and the second advertisement corresponding to the identified concept. 

147. (Previously Presented) The storage device of claim 141 , wherein the concept 
performance information for a concept includes one or more of selection information and 
impression information for advertisements presented with the document and corresponding to the 
concept. 
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148. (Previously Presented) The storage device of claim 141 , wherein the one or more 
code segments further comprise one or more code segments that, when executed, cause the at 
least one processor to determine a confidence measure for the advertisement performance 
information, the confidence measure representing a confidence level for the advertisement 
performance information relative to other advertisement performance information. 

149. (Previously Presented) The storage device of claim 148, wherein the one or more 
code segments that, when executed, cause the at least one processor to determine the confidence 
measure for the advertisement performance information comprise one or more code segments 
that, when executed, cause the at least one processor to determine a confidence measure for the 
advertisement performance information based on one or more of the age of data included in the 
advertisement performance information and the amount of the data included in the advertisement 
performance information. 

150. (Previously Presented) The storage device of claim 141 , wherein the 
advertisement for which the initial advertisement performance information is modified is 
included with the forwarded advertisements. 

151. (Previously Presented) The storage device of claim 141, wherein the 
advertisement for which the initial advertisement performance information is modified is 
different from the forwarded advertisements. 
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152. (Previously Presented) The storage device of claim 141, wherein the one or more 
code segments that, when executed, cause the at least one processor to forward, based on the 
modified advertisement performance information, the at least one of the one or more 
advertisements comprise one or more code segments that, when executed, cause the at least one 
processor to: 

compare the modified advertisement performance information for an advertisement to a 
threshold; 

determine that the modified advertisement performance information exceeds the 
threshold; and 

forward the advertisement based on the modified advertisement performance information 
exceeding the threshold. 

153. (Previously Presented) The storage device of claim 141, wherein the one or more 
code segments that, when executed, cause the at least one processor to: 

identify the one or more content concepts to which the document is relevant comprise one 
or more code segments that, when executed, cause the at least one processor to identify a first 
concept and a second concept to which the document is relevant; and 

determine that an advertisement is associated with the first concept and the second 
concept for the document. 



1 54. (Previously Presented) The storage device of claim 141 , wherein the one or more 
code segments that, when executed, cause the at least one processor to: 
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receive the initial advertisement performance information for the advertisements 
comprise one or more code segments that, when executed, cause the at least one processor to 
receive advertisement targeting information for the advertisements; and 

receive the concept performance information comprise one or more code segments that, 
when executed, cause the at least one processor to receive concept targeting performance 
information representing an aggregate performance of advertisements targeted according to the 
advertisement targeting information that were presented with the document based on the 
advertisement targeting information. 

155. (Previously Presented) The storage device of claim 141, wherein the one or more 
code segments further comprise one or more code segments that, when executed, cause the at 
least one processor to: 

identify a first advertisement that does not have initial advertisement performance 
information; 

determine that at least one of identified concepts for the document correspond to the first 
advertisement; 

receive concept performance information for the at least one of the one identified 
concepts corresponding to the first advertisement; and 

determine, based on the received concept information for the at least one of the identified 
concepts associated with the first advertisement, estimated advertisement performance 
information for the first advertisement. 
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156. (Previously Presented) The storage device of claim 155, wherein the one or more 
code segments that, when executed, cause the at least one processor to determine the estimated 
advertisement performance information for the first advertisement comprise one or more code 
segments that, when executed, cause the at least one processor to determine a weighted-sum of 
the content concept information for the at least one of the concepts corresponding to the first 
advertisement. 
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None. 
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